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Foreword 


At the outset of this work, we hypothesized that charter school oversight structures made it difficult for 
charter schools with nontraditional models to open, operate, and flourish. We worried that the trend toward 
increasingly standardized authorizing practices would stymie the diversity of models in the charter sector 
and limit families’ choices. Over the past year, we have sought to understand whether these hypotheses 
were true and what could be done. Of course, the world looks very different today than it did when we 
commenced this work. 


Before the pandemic, the phrase “nontraditional school models” may have brought to mind Montessori 
schools, competency-based education, and dual-language programs. Today, the phrase is more likely to 
bring to mind virtual instruction, hybrid homeschooling, and learning pods. Moreover, few schools today are 
operating a “traditional” model. The pandemic is forcing educators, policymakers, and funders to 
fundamentally rethink how we sustain student learning in the midst of unprecedented disruption. 


The lessons and recommendations surfaced in our research are more salient than ever and will help 
authorizers, school leaders, and others navigate another school year in the midst of a pandemic. In addition 
to the resources we initially created as part of our work on nontraditional schools, we developed a 
complementary toolkit that translates our findings into concrete guidance for school leaders and authorizers 
in the current moment. 


In the short term, we hope this toolkit helps school leaders and authorizers collaborate on their approaches 
to sustaining learning over the coming weeks and months. And in the long term, we hope this collection of 

resources will inform how schools, authorizers, and others balance autonomy and accountability in service 
of creating a diversity of high-quality options for students and families. 
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This toolkit for authorizers is accompanied by a policy 
report and a toolkit for school leaders 


This document 


Standardized or 
Customized? 
school Authorizers Can Better 
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Policy Report Authorizer Toolkit School Leader Toolkit 
The policy report provides an The authorizer toolkit outlines The school leader toolkit 
analysis and detailed findings potential strategies for outlines potential strategies for 
about the challenges and authorizers to consider in their the leaders of nontraditional 
opportunities of authorizing efforts to foster a diversity of schools to use when they work 
nontraditional charter schools, high-quality options, with with authorizers, including 
with recommendations for actionable templates, tools, and actionable templates, tools, 


authorizers, school leaders, frameworks to guide their work and frameworks 
funders, and policymakers 
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Introduction 


The ability to foster a diverse set of schools is an essential value proposition of the charter sector. The 
one-size-fits-all approach has proven to have limited utility, and the charter sector is well-positioned to foster 
a diversity of options that meet a variety of student needs; also, in a country as large as ours, the charter 
sector can accommodate a plurality of family preferences about the education they want for their children. 
Charter school authorizers, meanwhile, have a mandate to ensure school quality, and their standard 
measures of school quality can often be in tension with fostering school model diversity. 


Authorizers and school leaders must work together to ensure schools are accountable for strong 
outcomes but not stymied from implementing nontraditional models. 


In the accompanying policy report, we identify authorizing practices that can disadvantage schools with 
nontraditional models and may artificially limit their ability to open and expand. In the toolkit that follows, we 
provide authorizers with a set of considerations, options, and tools to guide how they work with school 
leaders to address these challenges and ensure their practices foster both the quality and diversity of 
charter school options. We provide parallel recommendations for school leaders in a separate toolkit. Our 
goal is to shed light on the importance of authorizing nontraditional charter schools and how the challenges 
of authorizing nontraditional schools manifest in practice, as well as to provide action steps for authorizers 
and school leaders who are able and eager to lead the field on these complex issues. 


Importantly, it is not our intent to provide an exhaustive list of all the scenarios that nontraditional schools 
and authorizers may face, nor to propose a “right” answer. Each authorizer must consider the approach that 
makes the most sense given their constraints and local contexts. Rather, we hope this toolkit: 


e Informs authorizers about the challenges and opportunities of nontraditional schools 
e illustrates options authorizers may consider for their approach to nontraditional schools 
e Shares useful frameworks for considering the benefits and trade-offs of these options 


Approach and methods (1 of 2): How do we define 
“nontraditional” charter schools? 


WMal-)a-mr- lcm: Aude (Mc: 1 a (21 8’ me) meter a Lele) Mi atleye(:] (MMV cM ol-tor-la Mole] mr-lar-lhVc)CmUVitamente)com lat-lamr- Me ley4-lamce stole) 
models recently identified and defined by the National Association of Charter School Authorizers (NACSA): 


Diverse by design Classical “No excuses” Vocational Alternative/credit recovery Arts 
Special education Inquiry-based Early college Policy Gifted STEM 
International/foreign language Blended/hybrid Military Virtual Single sex 


A subset of models provides particularly useful illustrations. For clarity and consistency, we 
focus on three models that often present challenges for authorizers; we exclude other models entirely. 


We focus on: We exclude: 

e Inquiry-based schools: Revolve curriculum and practices around e Schools designed to serve specific 
students “learning by doing” (e.g., Montessori, Waldorf, Reggio Emilia, student populations, such as 
and Expeditionary Learning models) students with autism, overage and 

e Dual language immersion: Provide instruction in English and a target under-credited students, or students 
second language across subjects as a central component of their mission in foster care. These schools also 

e Competency-based schools: Provide students flexibility in how they experience misalignment with 
demonstrate mastery of a particular skill or subject, allowing them to standard authorizing practices, but 
advance their learning independent of time spent on a subject merit a separate, thorough analysis 


Source: National Association of Charter School Authorizers (NACSA), Reinvigorating the 
Pipeline: Insights into Proposed and Approved Charter Schools, 2019. 6 


Approach and methods (2 of 2): How we identified challenges 
and strategies for authorizing nontraditional charter schools 


How did we identify charter schools’ Co) iYmel(o MU: M-JUlar-(ex-m elo) (-Jalir-]m-| 0) 0) cey-Leval-\-i ce) g 
challenges with authorizing? authorizers? 

To identify and understand how standard We identified four authorizers from across the 
authorizing practices can create pain country who have wrestled with whether and how to 
points for nontraditional charter schools, adapt their practices to foster more nontraditional 
we reached out to our networks and charter schools. We reviewed their data, analyzed 
solicited input from school leaders key documentation, and interviewed their staff. 
themselves. 

Ultimately, more than two dozen leaders COLORADO Colorado Charter School Institute 


CHARTER SCHOOL INSTITUTE 


of nontraditional schools shared 
specific examples of these challenges 
— as well as some of the ways in which 
they have been able to navigate them. 


State Charter Schools Commission 
of Georgia 


We pressure-tested these challenges with 


staff from eight different authorizers and DC District of Columbia Public Charter 
nine veterans of the charter sector. CHARTER School Board 


The State University of New York 
Charter Schools Institute 
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Current situation: Nontraditional schools and authorizers 
face challenges throughout the authorizing process 


Lever Challenges for authorizers 

e Authorizers lack familiarity with the model, requiring school leaders to make a stronger case for 
PNeye)i(er-litel a B their school than would otherwise be necessary and/or reducing likelihood of approval 
Ye) elce\e-lme-larems @ Authorizers have standard performance metrics that can conflict with schools’ instructional model 
red aFelacclalare, e Authorizers have standard performance metrics that do not fully capture schools’ distinct value 


e Authorizers allow schools to write additional metrics, but schools lack the expertise to do this well 


WKeYalixeyartare| e Authorizers use standard metrics that do not accurately reflect schools’ performance 

reviews e Authorizers use metrics that incentivize schools to adopt practices in conflict with their model 

e . Authorizers use standard performance metrics that are more likely to put nontraditional schools on 
the cusp between renewal and non-renewal 

High-stakes e Authorizers use standard rubrics for site visits and classroom observations, which may be 

reviews misaligned with schools’ instructional approach 


e Authorizers have set intervals for schools’ first high-stakes reviews that may negatively affect their 
_ performance, as nontraditional program designs may take more time to fully launch 


e ~ Authorizers make expansion decisions based on past performance on standard metrics, “which | may 
disadvantage schools in the expansion process 
e Authorizers lack familiarity with schools and require them to make a stronger case for expansion 


Model 
expansion 


e - Authorizers do not understand the model, or why performance looks different from other schools, 
Soft levers and may unnecessarily flag emerging concerns, with downstream impact on schools’ reputation 
e Authorizers may miss opportunities to highlight successes of the schools in their portfolio 


Individual challenges may seem small, but they accumulate. Ultimately, they limit the 


diversity of models available to students and families. 


Challenges & strategies: Authorizers can advance their 
work with nontraditional charter schools in three ways 


Challenge Strategies for Authorizers Resources & Tools 
F h chall h 
Limitations in knowledge, Be aware and build knowledge Sie ve 
expertise, time, and of specific models by ; 
: resources constrain how increasing access to data, Description of 
Capacity authorizers tailor their building expertise to use data BS sense: and 
approach to nontraditional effectively, and leveraging strategies, from the 
schools external partners perspective of the 
authorizer 
Develop or improve upon 
. ; performance metrics that can 
Serlnie aha accommodate a range of Tools, including 
Metrics with or ma Anel capture a sence models walle har aieamenet dats 
entiaiieral Shee model's paintetning' 3 nigh Par Ol — eo . 
value preposition excellence; complement support decision-— 
pee metrics with professional making and to mitigate 
judgment challenges 
Key stakeholders, including Build trust and ongoing . 
school leaders, authorizing communication with school Case studies wil) 


sample approaches of 
how some authorizers 
are implementing 
strategies to address 
challenges 


boards, families, and the leaders, board members, 
Ofeyinanlelalier-isielay) broader public, may be parents, and other 
unfamiliar with nontraditional stakeholders to proactively 
school models or how the convey value and explain 
authorizer approaches them approach 


Intended outcomes: Authorizers have the opportunity to 
foster a diversity of high-quality schools 


Authorizers have the opportunity and responsibility to ensure that all students can flourish, that 
diverse school models are equally available to all families, and that all schools are held 
accountable for providing a high-quality education. Authorizers must navigate the tension 
between maintaining consistent standards and fostering diverse models. This toolkit seeks 
to encourage and inform that work, and we offer recommendations for how authorizers and other 
stakeholders can work to find the optimal balance. Ultimately, this toolkit aims to support a 
charter sector in which more families have access to a diversity of high-quality options. 
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Authorizers can build their capacity to foster high-quality, 
nontraditional charter schools 


Capacity: Description 


Limitations in knowledge, expertise, time, and resources: 

Authorizers may not recognize or understand the value of nontraditional schools, or 
they may lack expertise in nontraditional schools’ models. They may not be aware of how 
the school’s distinct model could affect the school’s performance against standard 
metrics, and they may not have the additional capacity required to tailor their approach for 
nontraditional schools. 


This can have at least five consequences: 

1. Authorizers may discourage nontraditional schools from applying (or at least not 
encourage them to apply) for a charter in the first place 

2. Authorizers may be less likely to approve a charter for a model they don’t understand 

3. Authorizers may penalize or non-renew schools that are higher quality than is 
reflected on standard performance metrics or on other inputs considered in high-stakes 
reviews 

4. Authorizers may miss opportunities to encourage the growth of high-quality schools 

5. Authorizers may miss opportunities to elevate the excellence of nontraditional 
schools to stakeholders, including policymakers, communities, and families 
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There are several strategies for how authorizers can build 
their capacity 


Capacity: Tools —_—| 


To build their capacity, authorizers can: 


Build their awareness of nontraditional school models and the potential 
challenges they may experience in the authorizing process 


(2) Build capacity and expertise of authorizing staff 
(3) Inform the authorizing board of the staff's approach 


(4) Leverage expertise from partnerships and national associations 
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4) Build awareness: Authorizers can invest in their own 
knowledge of challenges specific to nontraditional models 


Capacity: Tools —_—| BTete) |S 
Authorizers can familiarize themselves with the challenges experienced by some of 
the more common nontraditional models 


Models Example Challenges for More Common Nontraditional Models 


e Pacing of achievement: Since this model emphasizes language instruction in multiple subject 
areas in a target second language, students’ proficiency rates may be lower on standardized 
tests conducted in English, especially in elementary grades 

e Achievement in other languages: Schools may not be rewarded for achieving their goals of 
language proficiency in the target second language 


e Multigrade classrooms: For multigrade classrooms, schools may have varying numbers of 
students in actual grade levels. Individual grade enrollment may not reflect overall demand, or 
outcome data may not reflect overall performance 


Niteltiiave=s-t-.en © Alignment with rubrics: Observation rubrics that reward teacher-student interactions are in 
Models tension with instructional components that encourage uninterrupted student-directed learning 

e Technology proficiency: Schools that actively avoid technology in the classroom may face a 
challenge on state assessments that are administered on computers, regardless of students’ 
knowledge/proficiency 


e Seat-time requirements: State and authorizer requirements on hours of exposure to a subject 
or length of time in a class conflicts with models that measure students’ ability to master a topic 

Competency- in ways independent of time or duration 

=r tt-TelfefelIy @ Learning progression: The sequence of instruction is different from other models, and 

therefore students may not learn the same content, in the same sequence, as is measured by 

standardized assessments 


DI UF-} ig Me-TateLer-le [= 
Models 
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4) Build awareness: Authorizers can ask questions to 
understand the differentiating features of a school model 


Capacity: Tools —_—| BTeye) |S 
Authorizers can build awareness by asking questions that apply to all school models 


Questions to ask 


9 Is this school model in demand from families? How 
“do you know? Do other schools have this model? 


4 What are the distinguishing characteristics of the 
“model? How do they support student success? 


4 Do you expect students in this model would achieve 
“proficiency at a different pace than others? Why? 


If students are expected to achieve proficiency over a 
longer timeline, what is that timeline? 


4 Do you anticipate that students will learn content in a 
“sequence different from that on state tests? Why? 


4 How do you measure student growth internally, to 
“inform continuous improvement? 


4 Does the school model rely on a particular style of 
* pedagogy? If so, what should we look for when 
observing student-teacher classroom interactions? 


Who to ask 


Experts in a particular school 
'uileXe{-) Mm igigolecelame)ecxci-1aic-lile)alow-lale| 
consultations with authorizer staff and 
board 


Tod aofo) Mi (-x-[e(-)e-mmel0laialem-l0liale)ay4]are| 
interviews, informal meetings, and 
school site visits 


Other authorizers who oversee 
sXe a (ele) smi tamcyiaalitclamaarerelsiis 


Partners who support the high-quality 
aa] e){=daaT=1al (lilo) ame) msve afore) magtele(=1i<) 


@ Build capacity: To build capacity, authorizers can 
contract, hire, or develop additional expertise 


Capacity: Tools —_| BTete) |S 
Authorizers can consider investing in the following roles: 


1. Ameasurement and evaluation expert can work with nontraditional schools to develop 
metrics that are meaningful, valid, and attainable indicators of school model success 


2. Adata analyst can conduct sophisticated analyses and execute different data needs 


3. Experts who are knowledgeable in specific school model(s) can support the authorizer’s 
review/assessment of nontraditional school quality 


4. A liaison can ensure strong communication with the authorizer’s board, as well as 
families, the community, and other key stakeholders 


The above roles can be hired internally, contracted externally, or developed among 
existing staff, depending on need and the availability of resources 


If an authorizer does not have the resources for the above investments, it could partner with 


another authorizer that does — either to augment its own capacity or to support nontraditional 
schools to transfer to another authorizer* 


*Authorizers that transfer schools to other authorizers must be mindful of the potential that 
“authorizer shopping” dilutes accountability. See NACSA, “Ending Authorizer Shopping,” 2019. 16 


GB) Inform the board: Authorizer staff can build buy-in and 
engagement of the authorizing board 


Capacity: Tools —_—| aTeye) |S 


By educating and informing their board about nontraditional schools, authorizer staff can 


build buy-in and alignment for their approach 


Objective of Authorizer Staff 


Educate board members 
rl orvolUimalelalie-lelid(elar-] moyeqalete)| 
raakeye (=) 3 


Report to board members 
r]ofelUi m-) 0) e) key: lela ce) 
oY{=e-1-1-1 fale malelatur-lelidteyare!| 
rYoq aYote) M el:Jace)auir-lalers 


DY =Y=) of-YaM dace) iu(ste(e (= Mey m eler-1Ke, 
ralcVanley=)e-m- Tare (ce) mm a=y-) ele) ale mm ce) 
board member concerns 


Authorizer staff should adapt these strategies depending on the role and level of involvement of 
their board, which can vary considerably je 


Strategies 


Describe differences between nontraditional school models 
Quantify demand for nontraditional schools, using waitlist data, 
surveys of local families, or community meetings 

Arrange site visits to nontraditional schools (within or outside of the 
jurisdiction) so board can observe different models in action 


Document different types of nontraditional schools and their 
distinguishing characteristics and inform board about their performance 
(e.g., include model information in annual report) 

Describe approach and rationale for overseeing nontraditional 
schools to ensure board understanding and alignment 

Present process for developing additional metrics, so board 
understands what metrics do/do not mean, especially for accountability 


Arrange for school model experts to speak/consult with the board 
Consider adding a board member with expertise in the most 
common/in-demand nontraditional school models 


@ Partnerships: Authorizers can leverage organizations and 
associations for needed expertise 


Capacity: Tools —_—| BTete) |S 


By leveraging partnerships and other existing resources regarding nontraditional school 


models, authorizers can avoid reinventing the wheel and minimize the need for increasing 


internal capacity 


Local 
Organizations 


IE likelarel 


Organizations 


Researchers, 
Peers, & 
Experts 


Regional charter networks that connect and support a range of charter schools (e.g., 
California Charter Schools Association, Northeast Charter Schools Network) 
Intermediary support organizations, funders, nonprofits, and thought leaders that 
influence local education (e.g., New Schools for New Orleans, Education Forward DC) 


National networks of charter school authorizers and/or school leaders (e.g., National 
Association of Charter School Authorizers, National Alliance for Public Charter Schools) 
National organizations dedicated to supporting high-quality implementation of 
nontraditional models 
o Inquiry-based schools: American Montessori Society, Association Montessori 
International/USA, National Center for Montessori in the Public Sector, EL Education 
o Dual language immersion/bilingual schools: Association of Two-Way & Dual 
Language Education; DC Language Immersion Project 
o Competency-based schools: Clayton Christensen Institute, The Learning Accelerator 


Other authorizers of nontraditional schools that can share and disseminate best 
practices, such as the SUNY Charter School Institute’s Active Ingredients pilot 
Measurement and evaluation experts that can create or improve metrics 

State or local universities that provide data/evaluation expertise 


Authorizers have implemented various strategies to build 
their capacity, knowledge, and expertise 


- Capacity: Case Studies _| Case Studies 


Authorizer Example 
; When a nontraditional school applies for a charter, PCSB supplements its review team 
7 with an expert in that area. Examples include: 
| aeren e — Hiring a language specialist to evaluate a dual language immersion school 
ee e Hiring a Montessori expert to review applications for Montessori schools 
ww BOARD 


To ensure robust and comprehensive data collection of multiple measures, SCSC of 
Georgia established strategic partnerships with the Georgia Department of Education 
and Georgia State University for data and analytics. Collaboration enables: 


e Calculation of metrics that control for student groups (e.g., economically 
disadvantaged) 

e Analyses of student-level data that control for student characteristics (e.g., date of 
enrollment, learning disabilities) 


Colorado CSI established an Alternative Education Campuses” Task Force to provide 
input on accountability issues, with a particular focus on optional measures. More than 25 
members of the task force include participants from: 


Colorado Department of Education 
Colorado League of Charter Schools 


e 
e University of Colorado 
e 
e 


COLORADO 
CHARTER SCHOOL INSTITUTE 
Colorado Charter School Institute 
School leaders, board members, and other education representatives 


* Alternative Education Campuses are focused on alternative student populations, not nontraditional school models. However, learnings and 
strategies can apply to nontraditional schools. Source: Colorado Charter School Institute, “Establishing Optional Measures,” spring 2018. 19 


Authorizers’ standard performance metrics can be imperfect 
or incomplete measures of nontraditional schools’ quality 


Metrics: Description _| Description 


Imperfect or incomplete performance metrics: 

When authorizers use standard performance metrics to assess school quality, those 
measures may conflict with a school’s instructional model. In some cases, standard 
performance metrics may not accurately reflect the performance of nontraditional 
schools. In others, they may not effectively capture the distinct value of a 
nontraditional school model, or how those features meet distinct student needs. 


This can have at least five consequences: 

1. Authorizers’ standard performance metrics can incentivize nontraditional schools to 
adopt practices in conflict with their model, in order to perform better 

2. When standard performance metrics do not accurately reflect nontraditional school 
performance, they are more likely to be on the cusp of renewal or non-renewal 

3. Performance metrics that do not capture nontraditional schools’ key features may 
prevent authorizers from celebrating a school’s high quality or learning from 
school features that meet distinct student needs 

4. Nontraditional schools that perform marginally worse on performance metrics may 
be less likely to receive approval for expansion 

5. If the community perceives a school as high quality, but it underperforms against 


standard metrics, it could undermine trust in the accountability system overall 7 


Authorizers can choose whether and how to adapt their 
standard performance metrics for nontraditional schools 


Metrics: Toots | a Key) ks 


Standard Performance Metrics 


Authorizers (ideally) outline a 
school’s performance metrics in the 
charter contract, and use those 
metrics to evaluate school quality. 
Many authorizers have standard 
metrics they use for all schools in 
their portfolios, which often include: 


Growth/proficiency on state tests 
Indicators of college and career 
readiness 

Student attendance and reenrollment 
rates 

Optional mission-specific metrics 
(rarely used effectively) 


However, these standard 
a atsa ta (ox- Wor: la Mmed a =y-14- Meg ar-l(-¥ale (os) 
Ko) mateyaiie-Celii(eyar-] mx a lere) ky 
(see slide 13) 


To address 
challenges 
nontraditional 
schools experience 
with standard 
performance 
metrics, authorizers 
have three options 


(1) To Od ake lave (sm comoit-larer-laemiU(-1ia (en 


Authorizer holds schools accountable for meeting 
standard metrics (often a summary calculation 
based on a weighted average of metrics) and 
does not consider any additional metrics in 
assessing school quality 


2) Create Additional Metrics 


Authorizer may work with schools to create 

additional metrics, which they can use: 

a) Informally, to educate themselves about the 
model and inform their professional judgment 

b) Formally, as additional metrics for 
accountability 


&) 104 aF-Tate(-MOl-Y-Me) me) r-laler-lacmiU(cida (en 


Authorizers hold schools accountable for meeting 
at least one standard metric; the school is 
assessed against each metric individually, not 
against the metrics collectively 


4) No change: Authorizers may choose to maintain existing 
metrics of school quality 


Metrics: Toots | eye) ks 


Watcia:W- lao eL-val-iii mr: lalemae-\e(-tre)ai-wice) mr-lUi tale) ay4-1e-ma Lome aloley-{-MmalelmceMedar-Vale (=m dalcyi manl-iia ler 


Benefits Trade-offs 

e It minimizes change management, with no e It may disadvantage or discourage nontraditional 
need to update processes or communication or schools, which could limit charter sector’s ability to 
establish buy-in from key stakeholders meet family demands 

e It maintains comparability of school quality e It may require authorizers to use more professional 
data, for ease of access and analysis by judgment when school models affect performance 
stakeholders, especially families against standard performance metrics, which can 

e It does not require additional capacity or introduce ambiguity into bright lines of accountability 
resources to develop or support the continued e It could undermine trust in the accountability 
use of additional metrics system, if the community perceives a school as high 


quality, but it underperforms against standard metrics 


Authorizers should assess this approach within their own contexts and constraints: 


Do their existing metrics set a minimum threshold that they believe all schools should meet, regardless of school 
model? Could standard metrics be adjusted to minimize the disadvantage to nontraditional schools? 


What nontraditional models are in demand (and how much demand) from families? Are there existing 
nontraditional schools, how do they perform against standard metrics, and what is their potential for expansion? 
How might the current size, quality, and politics of the charter sector affect how this approach is received? 
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Metrics: Toots | aero) ks 
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Benefits Trade-offs 


Additional metrics: Trade-offs depend on whether 
additional metrics are used to inform judgment ... 


ak towm oX:Valcviltomr- Tale mm tars\e (stro) acy 


Authorizers develop a better understanding of the e Authorizers require additional capacity for 


theory of change behind the school model complexity of data, analysis, and review 
Authorizers can quantify the value proposition e Authorizers limit comparability of data across 

of the school model, often tied to distinct needs of schools, requiring more nuanced communication 
students in the community with stakeholders 


Authorizers signal openness to nontraditional 
models, encouraging more to apply and open 


Examples: 


Proficiency rate in a second language (e.g., Standards-based Measurement of Proficiency) 
Social-emotional learning outcomes (e.g., Panorama student survey) 
Cohort-based outcomes (for mixed grade-level school models) 


Authorizers should assess this approach within their own contexts and constraints: 


How will the authorizer partner with schools to develop additional metrics? 
How will authorizers execute an effective change-management process, with families, community members, and 
their board? How will the authorizer communicate their approach to each audience? 


How will additional metrics inform their professional judgment in making recommendations and decisions about 
charter approval, renewal, and expansion? 

How will authorizers ensure that their staff and board continue to use additional metrics informally, as contextual 
information about the school model rather than as a formal metric for accountability? 

How might the current size, quality, and politics of the charter sector affect how this approach is received? 
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Goce) metrics: ... Or whether authorizers use 
additional metrics for purposes of formal accountability 


Metrics: Toots | eye) ks 


©) OF c-¥-lifate mr-lorelid(olar-lMatl=1talex-M ce) mm Ol-t-M [a mecxed alee) m-Lexexe]Ulale-ledi lin Mmar- tM ol-VaCclilcw-lalemte-Le(-treviiy 


Benefits Trade-offs 

e Enables more comprehensive evaluation of e Risks perception of inequity or lowering the bar 
school model effectiveness and value proposition if the use of additional metrics dilutes other metrics 

e Holds schools accountable for delivering the e Can create unintended incentives for school 
model they have promised to families and the leaders and staff to pay more attention to metrics 
community than to faithful implementation of the model 


e Metrics are harder to change when formalized 


Examples: 

e Proficiency rate in a second language (e.g., Standards-based Measurement of Proficiency, or STAMP) 
e Social-emotional learning outcomes (e.g., Panorama student survey) 

e Cohort-based outcomes (for mixed grade-level school models) 


Authorizers should assess this approach within their own contexts and constraints: 


How will the authorizer partner with schools not only to develop additional metrics, but also to continuously 
monitor school performance against the metrics? 


How will authorizers execute an effective change-management process, with families, community members, and 


their board? How will the authorizer communicate their approach to each audience? 

How will the authorizer adjust additional metrics over time, as needed, without undermining accountability? 
How will the authorizer hold schools accountable for missing/meeting additional metrics, and will that be 
different from how schools are accountable for missing/meeting standard metrics? 

How might the current size, quality, and politics of the charter sector affect how this approach is received? 
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Gy eewens) metrics: Authorizers that use additional metrics 
can partner with schools to define what metrics should be 


Metrics: Toots | eye) ks 


Some schools may need to first 
invest in articulating their model 


How does the school’s ... 


e curriculum? 

e instructional approach? 
e school culture? 

e approach to talent? 

e school schedule? 

e external engagement? 
e other? 


.. support its distinct 
outcomes? 


Example: 
Students will receive 50% of 
instruction in Spanish 


Note: >1 element of school model 
may support a distinct outcome 


+e Start Here 


VAY ats lars Inomialsmcve alele) ks 


intended student outcomes 
that are not captured by 
Sits alors Kom anlelalexsie 


What student-level 
outcomes related to ... 


specific knowledge? 
executive function? 
social-emotional health? 
cultural awareness? 
other? 


... does this school’s 
model enable/support? 


Students will develop 
fluency in Spanish 


How will the school use ... 


e assessments? 

e student diagnostics? 

e student work portfolios? 
e student/family surveys? 
e other? 


... to set targets and 
assess progress toward 
outcomes? 


Students take Standards- 
based Measurement of 
Proficiency (STAMP) 
assessment 


What metrics indicate the 
school is delivering distinct 
outcomes successfully? 


How much will the 
outcomes improve, on 
what timeline? 


In creating additional 
performance metrics, school 
leaders must set goals for 
what they plan to achieve 
for their students 


90% of students who are 
enrolled from kindergarten 
will achieve fluency in 
Spanish by third grade 
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Gene) metrics: Authorizers must ensure that 
additional metrics are valid, reliable, and attainable 


Metrics: Tools| i [eye) ks 


WELiCeli ays 


Definition A metric is valid if the instrument 
measures what it intends to 
measure 


Guiding e Does the instrument measure 
Questions what you want it to? 
Is the metric a worthwhile 
indicator of learning or 
progress? 
Is the metric difficult to 
manipulate or “game”? 


Example 


Reliability PNiclarleyii a, 


A metric is reliable if it produces A metric is attainable if it uses 
consistent results under similar sources of information that are 
conditions (or can be) available 


e ls there sufficient sample Is there a measurement 
size? instrument or source of data 
e ls the data collection process that is a suitable input for the 
consistent? metric? 
Is there limited potential for Is there a reliable way to 
bias in how data is collected collect data, now and in the 
and analyzed? future? 


EVI Ce ig Reliable? Attainable? 


To evaluate a school’s progress toward its mission of environmental 
education, a metric is based on the number of bottles collected during 


a recycling drive 


To assess the quality of instruction, an authorizer (or school) 
administers a classroom observation rubric; however, evaluators who 


observe the same lesson rate it differently 


To measure long-term outcomes, a metric is based on students’ earned 
annual income 20 years after high school graduation 


Sources: The Graide Network, “Importance of Validity and Reliability in 
Classroom Assessments,” 2018; US Government Accountability Office, 


“Assessing Data Reliability,” 2019; TKI, “Reliability and Validity.” 
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Gy eonene) metrics: Authorizers should be mindful of 
potential constraints for developing additional metrics 


Metrics: Tools | a Keye) ks 


Additional Metric Considerations: 


Is there sufficient sample 
size? 


Ti Is the metric consistently 
tracked over time? 


Does the metric use a 
consistent scale? 


Are there too many 
metrics? 


iM Is the metric informing 


actions and decisions? 


Schools with smaller enrollment or analyses with smaller sample 
sizes may result in larger fluctuations in outcomes. Data with a small 
sample size can be supplemented with additional longitudinal data 


Ideally, metrics do not (or rarely) change. Consistent metric definition 
and collection enables year-over-year comparisons and learning for 


both authorizers and schools 


While rubrics or characteristics that becane Saal may y differ bie school 


model, metrics that have the same scale enables easier comparisons 


across schools and models 


The number corn metrics Bdiepeies on the number fot saniponeals ina 
school model. Authorizers and school leaders should prioritize quality 


of metrics over auanilty; and focus on what measures matter most 


A metric is sfasadinailll sal itit it is used dior: an intended purpose, SHA 
to inform continuous improvement. Authorizers and school leaders 
should consider removing metrics that are no longer used to drive 
actions or decisions 
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@) 


Metrics: Case Studies 


eae Colorado CSI 


CHARTER SCHOOL INSTITUTE 


e The Performance Framework explicitly 
defines the measures by which CSI holds 
all charter schools accountable with 
regards to academic, financial, and 
organizational performance 


e In addition to the performance framework, 
schools can develop mission-specific or 
model-specific measures as supplemental 
analyses 


e The authorizer incorporates mission-specific 
and model-specific measures as 
supplemental analysis in its professional 
judgments 


‘The model-specific work allows us to look at what 
components work well for that school and what value they 
add, beyond the basic agreement of being a public school.” 


Additional metrics example: Colorado CSI partners with 
schools to develop additional metrics specific to the model 


Mission- and Model-Specific Measures 


Definitions: 

e Mission-specific indicators are tailored to an 
individual school’s mission and vary 
accordingly (e.g., a school with a mission of 
developing students into engaged citizens may 
have a metric around students’ participation in 
community service projects) 

e Model-specific indicators may apply to all 
schools that operate a similar model (e.g., all 
students in any dual language immersion 
school may have a metric around achieving 
proficiency in a second language) 


Approach: 

e Colorado CSI collaborates closely with schools 
to first understand their mission and model. It 
partners with the school leader to develop 
valid, reliable, and attainable metrics that 
capture the school’s desired outcomes; 
model-specific metrics developed through this 
process can be used across schools with 
similar models 
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GB) Change the use of standard metrics: Other authorizers 
provide flexibility within standard metrics 


Metrics: Tools eye) ks 


There are benefits and trade-offs for authorizers that set the same standards for all schools in their 
roXe) ance) |(ommeolelar-lax-Mil->.dlel(-MiaMmaledi Maalen MOlY-Mtel-1ia (er Mce)mr-Lexerolelaicleyiiia 


Benefits Trade-offs 


e Authorizers maintain comparability of metrics |e Authorizers and schools may need additional 


between schools capacity to complete additional data gathering, 

e Authorizers are less likely to penalize schools quality checks, and analysis 
on individual metrics that do not fit e Standard metrics still may not capture the value 
nontraditional models proposition of the school model 


e This can be perceived as lowering the bar for 
quality, if some metrics have lower rigor and create 
“loopholes” 


Examples: 

e An authorizer may create a “beating the odds” metric that accounts for student characteristics in growth measures, 
and renew the school’s charter even if it falls short on other metrics of overall growth and proficiency 

e An authorizer might renew the charter of a school whose students demonstrate college readiness on the ACT or 
SAT, regardless of performance on a high school graduation test 


Authorizers should assess this approach within their own contexts and constraints: 


Do metrics have comparable levels of difficulty, so they provide reasonably equal accountability metrics and do 
not, in effect, lower the bar for quality? 


How will the authorizer communicate with stakeholders about schools that meet some but not all metrics? Does 
the authorizer have sufficient capacity to implement nuanced communications? 


How might the current size, quality, and politics of the charter sector affect how this approach is received? 
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GB) Change the use of standard metrics example: SCSC of 
Georgia has standard metrics, but uses them differently 


Metrics: Case Studies _| Case Studies 


SCSC of Georgia Using Standard Metrics Differently 


How it works for schools: 
The SCSC of Georgia assesses performance 


is e Meeting at least one of the quality metrics 
based on school’s ability to outperform other 


is a sufficient indication of quality for a 


schools in its attendance zone* on one of six school to meet expectations. 
measures of academic performance: e While not designed specifically with 
nontraditional schools in mind, this 
1. College and Career Ready Performance flexibility provides a number of ways for 
Index (CCRPI) Content Mastery schools to demonstrate quality and has 
2. CCRPI Progress the effect of accommodating some 
3. CCRPI Grade Band Score nontraditional models. 
4. CCRPI Single Score How it works for the SCSC of Georgia: 
5. Value-Added Impact on student achievement ; 
6. “Beating the Odds” designation e The SCSC of Georgia partners with the 


Georgia Department of Education and the 
Georgia Governor’s Office of Student 

“We have many ways to assess innovation Achievement to access the data 

that yield performance results. Our wealth of necessary to calculate schools 


data allows us to have flexible ways to capture SCLIN, ang SEUSS NER etal 
ii State University to execute the analyses 
the success of schools. 


*Attendance zone comparison scores are generated at both the district and school level, assessing the 
performance of each school students would otherwise attend if the state charter school were not an option ou 


* 


Strategies & Considerations idole lanl oynod arate mc: 


Effective communication with key stakeholders is crucial for 
fostering nontraditional schools 


Communication: Description | Description 


School founders may not realize the authorizer’s openness to approving nontraditional 
schools, the authorizer’s board may not understand the authorizer’s approach to 
overseeing them, and families and the broader public may not have an accurate view 
of their value or performance. Authorizers might miss the opportunity to build 
relationships with nontraditional school leaders or recognize their successes. 


1. Authorizing staff may not be positioned to understand, articulate, or champion 
the value of nontraditional schools to stakeholders 

2. Authorizing boards may not be positioned to understand, articulate, or 
champion the authorizer’s approach to nontraditional schools 

3. School founders may be disincentivized from applying in the first place 

4. Nontraditional schools and authorizers will not have trust and rapport necessary 
to navigate challenges when they arise 

5. Families and the broader public may perceive nontraditional schools as lower 
quality than they are 
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To foster nontraditional schools, authorizers must 
effectively communicate with a variety of stakeholders 


Communication: Tools _| i [eye) ks 


Authorizers must communicate with a variety of 


stakeholders ... 


... With tailored objectives in 
mind 


Authorizing Staff 


Typically responsible for executing 
authorizing policies and practices, 
making recommendations to 
board, and engaging directly with 
school leaders 


Authorizing Board 


Often elected or appointed 
by public officials; often 
part-time, volunteer roles 


School Leaders 


Including founders, 
principals, and charter 
school board members 


Stakeholders 


Including families, 
students, and the broader 
public 


Ensure board understands and 
buys into both the value of 
nontraditional schools and the 
authorizers’ approach to holding 
them accountable 


Build trust, collaboration, and 
mutually productive 
relationships with leaders of 
nontraditional schools 


Inform families, communities, 
and public about the 
importance, diversity, and quality 
of all schools they oversee, 
including nontraditional options 
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First and foremost, there are best practices for how 
authorizers communicate with various stakeholders 


Communication: Tools _| 


These principles of communication are true for all authorizers but are particularly 
important for authorizers of nontraditional schools 
e Be proactive by sharing rationale and knowledge of nontraditional schools to 
address common questions 


e Build and lean on strong relationships with school leaders and other partners to 
collaborate on strengthening communication with stakeholders 


e Be evidence-based, and use research and knowledge from partner organizations, 
as appropriate 


e Be transparent about areas where you have developed a perspective from evidence 
and indicators, as well as areas where you are in a process of continued learning 


e Balance being comprehensive yet approachable to address questions for a wide 
audience; avoid jargon 


e Provide examples to ensure that abstract concepts are grounded in concrete 
illustrations of how nontraditional schools are different 
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4) Authorizers can communicate with their boards to build 
understanding of their approach 


Communication: Tools _| a Kere)(S 


Authorizing 
Board 


Authorizing 
Staff 


By communicating effectively, authorizing staff can ensure their 
board not only understands their approach to nontraditional 
schools, but is also able to articulate and champion the 


approach to other stakeholders 


Regardless of relationships between authorizing staff and authorizing boards, staff should 
communicate with boards about their approach to nontraditional schools by ... 


... eStablishing transparency 
from the onset. Authorizer staff 
should proactively share 

information with boards about the 


demand for nontraditional 
schools, the approach to school 
oversight, and any differences in 
how it intends to assess the 
quality of nontraditional schools 


... facilitating authentic ... document and regularly 
engagement. To build the report on progress. Authorizer 
board’s awareness and staff should include information 
knowledge of nontraditional on nontraditional schools in 
schools, authorizer staff should reports to the board — including 
invite board members to the approach to oversight as well 
participate in school site visits as summaries of models and 
and/or invite school model performance in the authorizer 
experts to brief board members portfolio 
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Gy eet: can communicate with school leaders to 
build trust and collaborative relationships 


Communication: Tools _| i eye) ks 


Authorizers can ... 
e Sponsor and disseminate research on family demand, including demand for 


To encourage potential 
school founders that the 
authorizer is welcoming of 
nontraditional school 
models ... 


To build relationships with 
school leaders early and 
often, to establish trust and 
rapport necessary to resolve 
challenges ... 


To partner with school 
leaders to convey value of 
nontraditional models to the 
authorizing board, families, 
and the broader public ... 


To facilitate peer-learning 
networks for school leaders 
to learn from each other ... 


nontraditional models 

Highlight successes of existing nontraditional schools on website and social media 
Describe authorizer’s approach to nontraditional school models in new school 
application orientation meetings, application templates, etc. 

Reach out to potential founders early in application process to discuss their models 


Authorizers can... 


Be transparent, including their rationale for their approach to nontraditional schools 
and any limits on authorizer flexibility, so school leaders know what to expect 

Make space for informal engagement, including meetings with school leaders or site 
visits, specifically to learn about the model 

Solicit feedback and create opportunities for schools to provide input, and ask for 
examples of when and how school leaders experience challenges 


Authorizers can... 


Invite leaders to share materials or make presentations about their model to the 
authorizing board 

Gather input from school leaders about information to include in communications 
with stakeholders (e.g., annual reports for authorizers’ portfolio or individual schools), 
such as descriptions of the model, school performance, and community demand 


Authorizers can... 


Look for common challenges for school leaders from various nontraditional school 
models to identify potential for sharing experiences and lessons learned 


on 


GB) Authorizers can communicate with families, 
communities, and public about diversity, quality of options 


Communication: Tools _| Bere) 


For families, communities, and the broader public, authorizers must carefully consider what 
messages they need to communicate, who the best messengers are, and the appropriate mode of 


communication. They must provide data in formats that are engaging and easy to navigate, and be 
careful to include information useful and accessible to families and the community; they must 
reinforce the authorizer’s commitment to ensuring high-quality schools. 


Messages Messengers Modes of Communication 


mo) arelaiieslelitce)arslmsxevalele)mantere (sits 


Breicificnont Reports and publications 


e.g., annual reports, school report cards 
AVdanamalelalir-\elit(elar-lmcvealele)-er-lK—) 


Tan) ofolaccTalt Authorizer website 


VAY aks lam areyalinsvellive)ars) move aveyte) is) e.g., interactive school directories 
currently exist 


AAVdare lm ate)a) (r= (ell re)ar=] move: alele)er- li) Third-party websites 


JF) (To Come)el—10 e.g., universal enrollment websites 


Wi at-mel0r-1linae)mare)alir-lelit(e)ar=) 


schools Public meetings 


(Me Bem al=t=l ul ale f-Me) almcver alefe) m= |e}e) i(er= tle] ale 
How the authorizer assesses testimony to oversight bodies 
quality 
SYoleir= 1m aal=ve lie] 


Whether/how quality measures Sige nitien earcoook 


elas MOlliislasyalt 
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Colorado CSI leverages peer-learning fellowships and 
technology to strengthen communications 


Communication: Case Studies | Case Studies 


Interactive Website 


— se 
CHARTER SCHOOL INSTITUTE 
Home About + CSISchools Parents v App 
Map Locations 
e Colorado CSI uses its interactive website as a Enter address or batitucl on Jone ondide og ety oLeietecricouty or postel codeine 
tool to communicate with families. On its website, 
families can filter schools by ratings, grade level, vi - 
location, and model to find schools that best meet Blended Leeming /Enrepreneuship 
F case ANIMAS HIGH SCHOOL AXIS INTERNAT 
iii _—_——_ sere Se 
e To enable peer learning and provide training, tan aden 
7 Personalized Learnin 
tools, and coaching to schools, Colorado CSI Pcesosed 


p-cde.force.com/? Improvement Plan: 
dcode=8001 &scode= 


Waldorf 


implements a School Improvement Fellowship. 
The fellowship strengthens communications and 
relationship-building between authorizer and School Improvement Fellowship 


schools and across peer school networks : ; 
Conducted over quarterly virtual and in-person 


o While fellowship is not exclusive to sessions, the School Improvement Fellowship 
nontraditional schools, it has been targets supports to schools that need to 
particularly valuable as a peer-learning develop the capacity to improve. The 
opportunity for leaders of nontraditional fellowship is intentionally an opt-in program — 
schools it is not punitive — with the intention of 


improving outcomes through capacity building. 


Source: Colorado Charter School Institute, “CSI 
Portfolio of Schools,” 2018-19. 39 


PCSB's informal meetings with school leaders and reports 
to the public on equity and quality help build trust 


Communication: Case Studies | Case Studies 


| CHARTER DC PCSB 
% BOARD 
e PCSB holds “board-to-board” meetings 
between authorizing board and boards of 
individual schools, establishing collaboration 
and lines of communication between 
oversight entities 


e PCSB publishes equity reports and 
school quality reports to inform school 
leaders, families, and the general public of 
school performance 


o While PCSB’s strategies are not 
exclusive to nontraditional schools, 
the programming, collaboration 
opportunities, and reporting 
transparency benefit all schools, 
including nontraditional schools 


Source: Office of the State Superintendent of 
Education, “Equity Reports,” 


“Board-to-Board” Meetings 


DC PCSB holds “board-to-board” meetings 
with the boards of charter schools. In these 
meetings, DC PCSB discusses items that 
may not have been included in the 
performance management framework but are 
an emerging area of concern. These 
meetings allow the authorizer to express 
concerns and discuss them candidly; they 
also allow school leaders a chance to discuss 
any misalignment between standard 
performance measures and the school’s 
nontraditional model. These informal 
opportunities for communication can support 
trust and collaboration between authorizers 
and schools — particularly important for 
navigating the challenges that can arise for 
nontraditional schools. 
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Appendix 


Authorizer profile: Colorado Charter School - 


COLORADO 


| Nn stitute CHARTER SCHOOL INSTITUTE 


Type of authorizer: Independent chartering board 
Year founded: 2004 

Jurisdiction: Colorado 

Number of schools: 40 

Number of students: 18,275 


Approach to Nontraditional Schools 

Colorado CSI expects all schools in its portfolio to meet standard measures of academic growth and 
achievement. At the same time, Colorado CSI approaches its work with an orientation toward 
understanding how schools measure their success, whether they are successful, and how Colorado 
CSI can help them improve. 


Colorado CSI acknowledges that it does not have all of the answers for measuring school quality. And 
it recognizes that if it makes high-stakes decisions without the correct information, it risks closing 
high-quality, innovative schools that would excel if the accountability standards comprised more 
nuanced or comprehensive measures. Therefore, it leaves room for staff to use professional judgment 
when evaluating schools, informed by both qualitative observations and quantitative data. 


Source: NAPCS, “A Growing Movement,” 2019; Colorado 
Charter School Institute, “Who We Are.” BELLWETHER 
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D¢ 
PUBLIC 
CHARTER 


SCHOOL 


Authorizer profile: District of Columbia Public | 
Charter School Board (PCSB) 


ws BOARD 


Type of authorizer: Independent chartering board 

Year founded: 1996 

Jurisdiction: Washington, D.C. 

Number of schools: 62 local education agencies, which operate 123 campuses 
Number of students: 43,911 students 


Approach to Nontraditional Schools 

DC PCSB uses standard monitoring review metrics, called performance management frameworks 
(PMFs), to report annually on charter school quality and to inform expansion decisions. (DC PCSB has 
four PMFs: 1) Early Childhood, Elementary School, and Middle School; 2) High School; 3) Adult 
Education; and 4) Alternative Accountability.) High-stakes reviews for all schools, which take place 
every five years, are based on each school’s goals (which are commonly the same as the metrics 
outlined in the PMF). 


DC PCSB recognizes that nontraditional schools are in high demand from parents and indicates that it 
is very open to those schools. It does not believe that its authorizing practices have hindered the 
success or growth of nontraditional schools, and believes that all schools, regardless of model, should 
be able to meet expectations of the PMF. 


Source: DC PCSB, “2019 Annual Report,” 2019. 
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Authorizer profile: State Charter Schools 
Commission of Georgia (SCSC) 


Type of authorizer: Independent chartering board 
Year founded: 2012 

Jurisdiction: Georgia 

Number of schools: 35 schools 

Number of students: 32,565 students 


Approach to Nontraditional Schools 

The SCSC of Georgia creates flexibility for nontraditional schools in how it uses multiple 
measures of school quality for accountability. In short, the SCSC of Georgia assesses 
schools against numerous individual metrics. Meeting at least one of the quality metrics is 
a sufficient indication of quality for a school to meet expectations. This flexibility provides 
a number of ways for schools to demonstrate quality and therefore helps accommodate 
nontraditional models. The SCSC of Georgia believes this approach balances the need 
for standard measures (all schools have the same set of measures to reach) and the 
need to provide multiple measures of success for nontraditional models. 


Source: State Charter Schools Commission of Georgia, “SCSC 


Mission”; Georgia Department of Education, “Georgia’s Charter 
Schools: 2018-2019 and 2019-2020,” 2019. BELLWETHER 
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Authorizer profile: The State University of oe eet 
New York (SUNY) Charter Schools Institute to te 


Type of authorizer: Higher education institution 
Year founded: 1998 

Jurisdiction: New York state 

Number of schools: 186 

Number of students: 104,000 


Approach to Nontraditional Schools 

SUNY CSI employs flexibility that is particularly beneficial to nontraditional school models in 
various parts of its authorizing process. SUNY CSI has some flexibility in how it uses data to 
inform its review process and the frequency of school renewals. The authorizer may take into 
account knowledge of a school’s specific context in review processes in addition to leading 
indicators of a school’s long-term success. SUNY CSI also has some flexibility when it comes 
to renewal timelines, such as for schools that may need more time to set up the unique 
components of their model. Finally, SUNY CSI acknowledges that non-academic measures of 
student progress may provide useful additional information about school quality. It is leading a 
pilot program, Active Ingredients, to develop and test alternative measures of school and 
student success. 


Source: University of the State of New York - New York State Education 
Department, “New York State Charter School Fact Sheet,” September 2019. A5 
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